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The  Research  and  Technology 
Organisation  (RTO)  of  NATO 


RTO  is  the  single  focus  in  NATO  for  Defence  Research  and  Technology  activities.  Its  mission  is  to  conduct  and  promote 
co-operative  research  and  information  exchange.  The  objective  is  to  support  the  development  and  effective  use  of 
national  defence  research  and  technology  and  to  meet  the  military  needs  of  the  Alliance,  to  maintain  a  technological 
lead,  and  to  provide  advice  to  NATO  and  national  decision  makers.  The  RTO  performs  its  mission  with  the  support  of  an 
extensive  network  of  national  experts.  It  also  ensures  effective  co-ordination  with  other  NATO  bodies  involved  in  R&T 
activities. 


RTO  reports  both  to  the  Military  Committee  of  NATO  and  to  the  Conference  of  National  Armament  Directors. 
It  comprises  a  Research  and  Technology  Board  (RTB)  as  the  highest  level  of  national  representation  and  the  Research 
and  Technology  Agency  (RTA),  a  dedicated  staff  with  its  headquarters  in  Neuilly,  near  Paris,  France.  In  order  to 
facilitate  contacts  with  the  military  users  and  other  NATO  activities,  a  small  part  of  the  RTA  staff  is  located  in  NATO 
Headquarters  in  Brussels.  The  Brussels  staff  also  co-ordinates  RTO’s  co-operation  with  nations  in  Middle  and  Eastern 
Europe,  to  which  RTO  attaches  particular  importance  especially  as  working  together  in  the  field  of  research  is  one  of  the 
more  promising  areas  of  co-operation. 


The  total  spectrum  of  R&T  activities  is  covered  by  the  following  7  bodies: 


•  AVT 

•  HFM 

•  1ST 

•  NMSG 

•  SAS 

•  SCI 

•  SET 


Applied  Vehicle  Technology  Panel 
Human  Factors  and  Medicine  Panel 
Information  Systems  Technology  Panel 
NATO  Modelling  and  Simulation  Group 
System  Analysis  and  Studies  Panel 
Systems  Concepts  and  Integration  Panel 
Sensors  and  Electronics  Technology  Panel 


These  bodies  are  made  up  of  national  representatives  as  well  as  generally  recognised  ‘world  class’  scientists.  They  also 
provide  a  communication  link  to  military  users  and  other  NATO  bodies.  RTO’s  scientific  and  technological  work  is 
carried  out  by  Technical  Teams,  created  for  specific  activities  and  with  a  specific  duration.  Such  Technical  Teams  can 
organise  workshops,  symposia,  field  trials,  lecture  series  and  training  courses.  An  important  function  of  these  Technical 
Teams  is  to  ensure  the  continuity  of  the  expert  networks. 

RTO  builds  upon  earlier  co-operation  in  defence  research  and  technology  as  set-up  under  the  Advisory  Group  for 
Aerospace  Research  and  Development  (AGARD)  and  the  Defence  Research  Group  (DRG).  AGARD  and  the  DRG  share 
common  roots  in  that  they  were  both  established  at  the  initiative  of  Dr  Theodore  von  Karman,  a  leading  aerospace 
scientist,  who  early  on  recognised  the  importance  of  scientific  support  for  the  Allied  Armed  Forces.  RTO  is  capitalising 
on  these  common  roots  in  order  to  provide  the  Alliance  and  the  NATO  nations  with  a  strong  scientific  and  technological 
basis  that  will  guarantee  a  solid  base  for  the  future. 
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OVERVIEW 


The  SAS-081  Symposium  on  (( Analytical  Support  to  Defence  Transformation”  took  place  in  Sofia, 
Bulgaria  in  the  period  26-28  April  2010.  It  brought  together  90  participants  from  19  NATO  and  partner 
countries.  The  symposium  final  programme  featured  33  presentations,  structured  around  five  main 
themes:  General  topics;  Defence  planning;  Acquisition;  Concept  development  and  experimentation;  and 
Analysis  of  current  operations.  31  of  the  scheduled  presentation,  including  two  keynote  presentations  were 
made,  each  followed  by  a  vigorous  discussion. 

This  Technical  Evaluation  Report  summarises  core  ideas,  identifies  trends,  good  practice  and  gaps  in 
analytical  support  of  defence  transformation.  It  provides  an  overview  of  the  discussions  and  presents 
recommendations  to  NATO  and  national  authorities  and  the  leadership  of  their  analytical  organisations. 


1.0  INTRODUCTION 


The  symposium  addressed  the  contribution  of  analysis  to  defence  transformation.  It  provided  a  forum  for 
military  and  civilian  officials  to  express  their  transformational  needs  to  the  analytical  community,  to  share 
information  amongst  analysts,  and  to  stimulate  new  ideas  and  initiatives  on  how  analysis  can  enhance 
transformation.  Practitioners  and  analysts  were  able  to  discuss  experiences  from  the  implementation 
methods  and  tools  and  latest  research  results  in  support  of  transformation.  The  symposium  facilitated  the 
enhancement  of  the  body  of  knowledge  of  related  concepts,  methodologies,  methods,  and  tools,  the 
implementation  of  objective,  rational  decision-making  frameworks,  and  the  identification  of  key 
implementation  challenges  and  priority  research  areas. 

This  report  preserves  the  structure  of  the  symposium  programme,  with  slight  reformulation  of  the  main 
topics  to  reflect  the  actual  presentations  and  highlights  of  the  discussion.  Thus,  the  following  five  sections 
of  the  report  reflect  the  themes  of  the  five  symposium  sessions.  Section  2  presents  general  analysis  issues 
with  focus  on  main  transformation  areas  and  their  intersections,  including  the  role  of  partnerships,  and  the 
interaction  between  analysts  and  decision  makers.  Section  3  outlines  the  main  challenges  in  providing 
analytical  support  to  current  and  future  defence  planning.  Section  4  treats  the  analytical  challenges  in 
providing  for  efficient  and  well  coordinated  development  of  main  capability  components,  while  Section  5 
looks  at  the  advances  in  concept  development  and  experimentation.  Section  6  treats  current  challenges  in 
providing  analytical  support  to  ongoing  operations,  including  the  enhancement  of  lessons  learned 
processes.  The  final  section  presents  an  attempt  to  formulate  some  enduring  challenges  to  the  provision  of 
analytical  support.  The  report  concludes  with  selected  recommendations  to  leadership  of  the  defence 
establishments  and  senior  members  of  the  NATO  analytical  community  that,  in  the  opinion  of  the  author 
of  this  report,  may  serve  to  organise  future  studies  in  support  of  transformation. 
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2.0  GENERAL  ANALYSIS  ISSUES 

This  section  presents  recent  senior  views  on  defence  transformation,  the  main  transformation  areas  and 
linkages  among  them,  roles  of  the  analytical  community,  and  some  recommendations  on  shaping  OA 
research  programmes. 

2.1  Transformation  Challenges 

There  is  no  single,  generally  accepted  definition  of  defence  transformation.  Contributors  to  the  symposium 
used  different  definitions,  referring  to  NATO’s  new  missions,  operational  concepts,  capability 
requirements,  organisational  structures,  training  and  force  development  approaches. 

The  2010  edition  of  the  NATO  Glossary  of  Terms  and  Definitions  defines  transformation  as  a  “ continuous 
and  proactive  process  of  developing  and  integrating  innovative  concepts ,  doctrines  and  capabilities  in 
order  to  improve  the  effectiveness  and  interoperability  of  military  forces"  [1].  In  a  recent  discussion  on  the 
new  NATO  Strategic  Concept,  Secretary  General  Rasmussen  said  that  the  Alliance  transformation  is  about 
improving  NATO’s  working  methods  and  preparing  better  for  the  future: 

We  must  face  new  challenges.  Terrorism,  proliferation,  cyber  security  or  even  climate  change  will 
oblige  us  to  seek  new  ways  of  operating.  And  in  a  time  of  financial  and  budget  constraints,  we 
need  to  maximise  our  efficiency  within  limited  resources.  [2] 

During  the  same  discussion,  Dr.  Madeleine  K.  Albright — leader  of  the  group  of  experts  that  prepared  a 
report  to  the  North  Atlantic  Council  with  analysis  and  recommendations  on  the  new  NATO  Strategic 
Concept — emphasised  that  NATO  must  adapt  to  the  changing  times: 

NATO  can  neither  slow  the  pace  of  change  nor  ameliorate  the  quick-silver  quality  of  modern 
events.  But  if  Allies  are  truly  dedicated  to  their  shared  tasks,  it  can  be  a  reliable  and  predictable 
means  for  coping  effectively  with  unpredictable  threats.  [2] 

Uncertainty  is  also  emphasised  by  US  Secretary  of  State  Hillary  Clinton,  stating  in  February  2010  that 
“we  face  a  new  strategic  landscape.  New  technologies,  new  adversaries,  and  new  ideologies  threaten  our 
security.  And  once  again,  there  is  little  certainty  about  the  future”  [2]. 

Reviewing  some  historical  examples,  Roger  Forder  emphasised  in  his  plenary  report  that  the  current  wave 
of  transformation  is  not  as  unique  as  it  may  seem  [P02].  On  many  occasions,  leaders — utilising  new 
technologies,  innovative  strategies,  or  combination  of  the  two — have  been  able  to  achieve  dramatic 
improvements  in  the  face  of  novel  threats  or  to  take  advantage  of  emerging  opportunities.  Accepting  that 
as  a  historical  fact,  for  the  purposes  of  the  current  report  we  emphasise  three  features  of  the  current  wave 
of  defence  transformation: 

The  first  feature  is  agility.  In  the  face  of  uncertainty,  alliances,  nations  and  defence  establishments  have  to 
become  more  agile,  to  be  able  to  adapt  quickly  to  changing  security  requirements  and  resource  constraints, 
to  exploit  technological  opportunities,  to  take  advantage  of  new  business  models.  Agility  turns  into  a 
capability  of  its  own  right.  Defence  establishments  face  the  challenge  of  introducing  capability  planning 
and  development  processes,  resource  allocation  norms,  and  organisational  culture  that  enhance  the  ability 
of  the  organisation  to  adapt  to  changing  circumstances. 

The  second  feature  is  efficiency.  It  is  not  enough  to  demonstrate  that  military  forces  are  effective,  i.e.  that 
they  are  able  to  conduct  successfully  the  range  of  operations  envisioned  in  alliance  and  national  defence 
policies.  Parliaments  and  the  public  have  to  be  convinced,  that  the  resources  allocated  to  defence  are 
utilised  in  the  most  efficient  manner.  Defence  establishments  are  under  constant  pressure  to  streamline 
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organisational  processes,  to  introduce  novel  business  models,  to  benchmark  their  performance  in  the 
development  of  all  capability  components  to  best  practice,  doing  all  that  in  a  way  that  is  fully  transparent 
to  decision  makers  in  government  and  parliament.  On  the  other  hand,  decision  making  processes  in 
democracies  may  be  rather  complex,  thus  posing  an  additional  challenge  to  achieving  agility  of  defence 
establishments,  in  particular  in  times  of  budget  cuts  as  in  the  current  economic  and  financial  crisis. 

Third  is  the  role  of  cooperation  and  partnerships.  NATO  nations  cooperate  in  various  formats,  in  some 
cases  including  also  partner  nations,  in  procuring  defence  capabilities.  Also,  NATO  interacts  with  partner 
countries  in  operations  in  many  hot  spots  around  the  globe.  Thus,  interoperability  remains  a  challenge,  and 
that  includes  terminology,  decision  making  principles  and  processes,  operating  procedures,  technical 
standards,  organisational  culture,  etc.  In  addition,  what  matters  are  not  only  the  operational  capabilities, 
but  also  the  institutional  capacity  of  partners  willing  to  share  with  NATO  the  responsibilities  for  their  own 
security,  for  enhancing  regional  security  and,  in  some  cases,  addressing  global  security  challenges. 

2.2  Transformation  Areas  and  Main  Analytical  Links 

NATO  nations,  as  a  rule  together  with  partners,  are  involved  in  diverse  operations.  The  most  prominent 
among  them  are  the  operations  to  counter  terrorist  networks,  potentially  with  a  global  reach.  The 
understanding  of  these  operations  evolves,  and  that  is  seen  in  designations  like  “irregular  warfare,” 
counterinsurgency  (COIN)  operations,  “hybrid  warfare,”  Stability,  Security,  Transition,  and 
Reconstruction  (SSTR)  operations,  etc.  Both  policy  makers  and  operational  planners  understand  well  that 
the  desired  end  state  in  such  operations  cannot  be  achieved  with  kinetic  effects  alone.  To  be  successful,  the 
military  forces,  cooperating  in  a  network  manner  with  other  government  agencies,  international,  non¬ 
governmental  and  business  organisations,  have  to  address  root  causes  and  provide  certain  effects  on  the 
opponents,  their  support  base,  and  the  population.  Hence,  two  concepts  attract  the  prevailing  attention  of 
decision  makers  and  analysts  -  effects  based  approach  to  operations  and  network  enabled  capabilities. 

Both  concepts  impact  the  operations  planning  process.  They  also  drive  the  capability  planning  process, 
and  the  capability  requirements  review  in  particular.  Operational  concepts  impact — albeit  indirectly — the 
processes  of  delivering  capabilities  through  coordinated  management  of  organisational  structures  and 
processes,  people,  materiel,  infrastructure,  and  training.  Finally,  these  and  other  supporting  concepts  are 
refined,  tested,  and  eventually  validated  in  the  process  of  Concept  Development  and  Experimentation 
(CD&E). 

All  these  were  addressed  by  symposium  participants  as  stand  alone  issues,  but  also  in  their  interaction. 
The  symposium  also  addressed  the  technological  aspects  of  transformation,  going  beyond  the  analytical 
challenges  of  procurement  and  R&D  management.  New  technologies  are  the  most  frequent  driver  of 
defence  transformation,  although  it  is  not  the  only  one  nor  is  it  sufficient.  Of  particular  current  interest  is 
the  topic  of  non-lethal  weapons  (NLW),  defined  in  the  1999  NATO  NLW  policy  as  weapons  “explicitly 
designed  and  developed  to  incapacitate  or  repel  personnel,  with  a  low  probability  of  fatality  or  permanent 
injury,  or  to  disable  equipment,  with  minimal  undesired  damage  or  impact  on  the  environment.”  Analysts 
agree  that  the  challenges  and  opportunities  created  by  NLW  technologies  are  most  appropriately  studied 
within  context  change,  e.g.  through  alternative  futures  analysis,  as  well  as  along  with  relevant  changes  in 
concepts,  organization  or  technology,  e.g.  implementing  ideas  from  trend  analysis  and  game  theory  [P  28]. 

2.3  Roles  of  the  Analytical  Community  and  Interactions  with  End  Users 

The  symposium  participants  agreed  that  the  field  of  Operational  Analysis  (OA)  can  be  considered  to 
include  those  professionally  trained  analysts  that  are  experts  in  applying  rigorous,  structured,  scientific 
methods  to  understand  complex  problems  to  assist  decision-makers  within  the  military  domain  with 
independent  advice  [P  35].  This  analytical  community  has  a  role  in  addressing  all  transformation 
challenges.  It  not  only  supports  the  respective  decision-making  processes,  but  also  serves  as  a  pool  of 
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knowledge  and  experience  and  a  conduit  of  ideas,  critical  evaluation  and  good  practice  across 
transformation  areas.  For  example,  on  the  practitioners’  side,  the  communities  of  operations  planners, 
capability  planners,  acquisition,  infrastructure,  programme  managers,  trainers  and  experimenters  are  rather 
distinct,  and  the  military  officers  are  assigned  in  respective  positions  for  a  relatively  short  period.  On  their 
side,  and  in  addition  to  traditional  decision  support  roles,  analysts  are  often  assigned  for  longer  terms,  they 
are  used  to  exchange  information  with  peers  from  national  OA/OR  organisations  and  internationally,  to 
benchmark  experience  and  facilitate  the  introduction  of  best  practice. 

Notwithstanding  these  advantages,  the  interaction  with  end  users — primarily  decision  makers — is 
particularly  challenging.  Often  analysts  do  not  have  the  luxury  of  intense  observation  and  undisturbed 
contemplation.  Instead,  they  have  to  tailor  ruthlessly  the  analysis  to  the  time  scales  and  method  of  work  of 
the  decision  maker.  And  that  is  particularly  important  for  advice  in  theatre.  Roger  Forder  calls  this 
‘tyranny  of  time’,  noting  that  the  utility  of  the  analytical  tool  is  tied  to  time.  As  a  result,  advice  is  often 
sub-optimal.  Even  so,  based  on  his  rich  experience  he  counsels  that  “a  timely  advice  is  better  than  no 
advice”  [P  02]. 

For  years  analysts  have  been  able  to  learn  from  the  experience  of  their  professional  community  -  what 
works,  what  does  not,  how  to  interact  with  decision  makers,  and  so  on.  One  observation  from  the 
symposium  discussion  raises  concerns  in  that  respect  -  recent  organisational  changes  in  NATO  HQs  lead 
to  dispersion  of  operational  analysis  and  may  not  guarantee  the  professional  standing  and  development  of 
adequate  OA  practices  [P  35]. 

Han  de  Nijs  from  ACT  emphasises  that  there  is  a  large  and  established  body  of  people  whose  profession  is 
the  application  of  scientific  and  structured  analysis  to  support  decision-making  in  various  fields  in 
defence.  NATO  and  nations  can  benefit  from  a  well  established  professional  community  with  the 
following  main  goals  [P  35]: 

•  Promote  a  professional  identity  that  is  recognised  by  customers  and  self-identified; 

•  Education,  training  and  professional  development; 

•  Networking; 

•  Collaboration  on  a  project  basis; 

•  Technical  development,  to  include  identification  and  definition  of  shared  theories  and  concepts; 
best  practices  and  setting  technical,  ethical  standards  and  values;  and  mechanism  to  share  (and 
validate)  work  of  common  interest  to  members  of  the  community; 

•  Identification  of  issues  of  future  importance; 

•  Community  support  to  include  representation,  moral  and  social  support,  rewarding  and 
recognising  achievement. 

2.4  Enhancing  NATO  Partnerships 

The  community  of  analysts  within  NATO  and  nations  may  serve  as  an  efficient  conduit  of  good  practice 
and  other  know  how  to  more  recent  NATO  members  and  to  willing  partner  countries.  The  exchange  of 
information  and  experience  facilitates  the  increase  of  interoperability  within  the  alliance  and  with  partners 
through  better  understanding  of  context,  terminology,  methods,  measures,  and  implementation  challenges. 
That  encompasses  all  main  areas  of  analytical  support,  including  support  to  operations,  lessons  learned, 
capability  planning,  capability  development,  concept  development  and  experimentation.  Particularly 
important  for  new  NATO  members  is  the  learning  experience  of  how  to  organise  OA  research  and  to 
provide  analytical  support. 
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Interactions  of  analysts  help  to  set  the  stage  for  multinational  capability  development  and  R&D  projects 
among  NATO  and  other  partner  networks,  thus  increasing  efficiencies  and  alleviating  the  impact  of 
shrinking  defence  budgets  in  the  current  economic  and  financial  crisis. 

Norway  provides  an  example  of  analysts  assisting  the  establishment  of  analytical  capacity  in  support  to 
defence  planning  in  partner  countries.  Such  efforts  improve  communication,  contribute  to  defence 
institution  building  in  partner  countries,  and  may  prove  to  be  a  very  cost  efficient  contribution  to  the 
increase  of  interoperability  of  partner  forces  and  their  contribution  to  NATO  operations. 

This  is  just  a  sample  of  examples,  showing  that  the  analytical  community  contributes  to  all  main  areas  of 
the  Alliance  transformation. 

2.5  Shaping  OA  Research  Programmes 

Symposium  presentations  and  the  discussion  facilitate  the  understanding  of  how  to  shape  an  OA  research 
programme.  One  recurring  topic  was  the  search  for  adequate  definitions  and  taxonomies.  According  to 
Sue  Collins  and  Simon  Purton,  NATO  definitions  are  frequently  ambiguous  and  vague,  derived  through  a 
consensus  among  parties  with  different  views  [P08].  This  has  hindered,  inter  alia,  the  analysis  of 
requirements  for  expeditionary  operations  in  the  S AS -075  study.  The  authors  of  the  report  have  used 
General  Morphological  Analysis  as  a  method  to  structure  this  ‘wicked’  problem  and  to  consider  potential 
solutions.  They  have  created  an  interactive,  dynamic  tool  based  on  a  Morphological  Table  to  capture 
expert  opinion.  Then  they  eliminate  non-valid  combinations  of  scalars  in  the  table  that  may  result  from 
logical  contradictions,  empirical  inconsistencies,  or  normative  constraints.  Data  mining  techniques,  i.e.  a 
classification  tree,  and  standard  statistical  analysis  software  are  used  to  identify  predictors  and  build  rule 
sets  that  model  different  problem  definitions  or  solutions  to  the  research  question.  This  method  is 
considered  of  potentially  wide  utility  within  NATO  and  national  capability  development  domains  [P  08], 
thus  increasing  the  transparency  within  NATO. 

Anton  Minkov  from  Centre  for  Operational  Research  and  Analysis  at  Defence  Research  and  Development 
Canada  provided  another  example  of  analytical  support  to  increasing  transparency  of  national  and  Alliance 
planning  [P  05].  The  author  presented  an  analytical  framework  for  assessing  security  and  regime  stability 
that  is  based  on  a  correlation  between  several  demographic  stress  factors  and  accounts  for  GDP  per  capita 
and  unemployment  rates,  as  well  as  for  the  presence  of  ethnic  or  sectarian  tension.  Studying  open  source 
data  for  15  Middle  East  countries,  the  author  assesses  the  probability  of  demographic  growth  to  result  in 
regime  changes  or  democratic  reforms  based  on  a  correlation  between  the  strength  of  the  youth  cohort  and 
the  regimes’  ability  to  retain  power  [P05].  This  can  be  seen  as  an  example  how  rigorous  analytics  can 
make  planning  assumptions  and  scenarios — planning  situations,  alternative  futures,  or  both —  and  thus  the 
defence  planning  process  more  transparent. 

The  symposium  discussion  increased  the  understanding  of  analytical  requirements.  OA  research  managers 
and  senior  analysts  need  to  understand  current  and  anticipate  future  requirements  in  order  to  shape 
national  OA  research  programmes.  One  trend  is  the  increasing  demand  for  soft  OA  support,  addressed 
briefly  in  section  7  of  this  report.  In  addition,  they  need  to  keep  a  picture  of  the  potential  contributions  of 
the  analysts  vis  a  vis  the  currently  available  analytical  capabilities  [P  02].  Given  that  demand  for  analytical 
support  usually  exceeds  the  available  capacity,  a  better  understanding  of  gaps  in  analytical  capability 
could  lead  to  an  increase  of  the  exchange  among  peers  from  allied  and  partner  nations  and  the 
international  cooperation  in  OA/OR  programmes  and  projects. 


3.0  ANALYTICAL  SUPPORT  TO  DEFENCE  PLANNING 

The  presentations  and  the  discussions  during  the  symposium  supported  the  thesis  that  Capability-Based 
Planning  (CBP)  has  already  turned  into  a  “gold  standard”  for  the  defence  planners  in  NATO,  NATO 
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nations,  and  many  of  the  partner  countries.  CBP  is  already  solidly  situated  at  the  core  of  national  force 
development  processes.  In  all  presented  instances,  CBP  approaches  utilise  scenarios,1  or  “planning 
situations”  in  the  official  NATO  language,  attempt  to  distinguish  between  capability  requirements  and 
respective  organisational  and/or  material  solutions,  and  seek  the  answers  within  more  or  less  strict 
resource  constraints  framework  with  understanding  of  risks.  This  trend  was  boosted  with  the  introduction 
of  the  new  NATO  Defence  Planning  Process  (NDPP),  its  methods,  tools,  and  terminology.2 

This  section  of  the  report  draws  heavily  on  Dr.  Alan  Campbell’  report  [P  09],  summarising  the  findings  of 
the  workshop  on  the  Analytic  Implications  of  the  NATO  Defence  Planning  Process ,  conducted  by  the 
SAS-081  Specialist  Team  at  the  NATO  Consultation,  Command  and  Control  Agency  in  The  Hague,  The 
Netherlands,  2-A  March  2010. 

3.1  The  Use  of  Scenarios 

The  use  of  scenarios  is  already  recognised  as  being  indispensible  in  capability -based  planning.  Scenarios 
serve  three  main  purposes. 

First,  through  scenarios  defence  policy  makers  and  planners  make  explicit  various  assumptions  underlying 
national — as  well  as  allied — security  and  defence  policies  and  high-level  guidance.  Typically,  high-level 
guidance  defines  roles  and  missions  of  the  armed  forces  and  provides  broad  description  of  future  security 
environment,  without  explicitly  defining  defence  planning  scenario  sets.  Scenario  sets  are  developed 
subsequently  to  span  the  spectrum  of  roles  and  missions  of  the  armed  forces,  as  defined  in  high-level 
policy  or  guidance  [P  09]. 

Second,  the  use  of  scenarios  enhances  the  credibility  of  the  planning  effort.  It  provides  a  framework  that  is 
considered  adequate  for  exercising  military  judgement,  offers  a  quantification  tool,  and  maximises 
analytical  rigour  and  traceability  [P  09]. 

Third,  scenarios  provide  a  common  framework  for  planning  across  different  defence  domains,  or 
capability  components,  and  enhance  the  understanding  of  interactions  among  capabilities.  Thus,  the  use  of 
scenarios  is  helpful  in  penetrating  organisational  boundaries  and  overcoming  service  biases. 

This  feature  is  potentially  important  in  a  broader  security  setting,  and  attempts  have  already  been  made  to 
utilise  it.  One  application  area  is  the  allocation  of  requisite  capabilities  and  the  coordination  of  capability 
development  among  national  security  sector  organisations  [5].  Likewise,  in  operations  with  the  military  in 
a  lead  role,  such  as  the  ongoing  operations  in  Afghanistan,  the  use  of  common  scenarios  in  the  planning 
phase  provides  for  better  understanding  of  requirements  to  and  by  all  contributors  involved  in  the 
operation,  including  international  organisation,  aid  and  other  non-governmental  organisations,  and  private 
security  companies. 

Nations  with  well  established  defence  planning  processes  recognise  the  advantage  of  using  generic 
scenarios  to  reduce  political  sensitivity  and  potential  security  classification.  Nevertheless,  the  benefits  of 
scenario-based  assessments  are  maximised  with  the  use  of  real-world  scenarios,  thus  increasing  fidelity, 
avoiding  the  need  to  justify  fictitious  settings  and  assumptions,  and  assuring  greatest  degree  of  military 
credibility.  That  allows  also  increase  in  the  level  of  realistic  detail  in  design  to  a  degree  that  turns  planning 
scenarios  into  pseudo-operational  scenarios  [P  09]. 
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However,  even  though  both  the  design  and  the  selection  of  planning  scenarios  are  based  on  intelligence 
assessments  and  foresight,  the  use  of  real-world  scenarios  may  fixate  the  planners  on  current  challenges. 

In  its  origins,  the  use  of  scenario  sets  has  been  intended  to  help  planners  deal  with  uncertainty.  There  is  a 
growing  understanding  that  that  may  not  suffice  in  a  force  development  environment  characterised  by 
“deep  uncertainty.”3  Therefore  some  countries  investigate  a  concept  with  two  levels  of  scenarios  - 
scenarios  describing  alternative  futures  at  the  upper  level  and  one  or  more  'point  case’  planning  scenarios 
on  the  lower  level  corresponding  to  each  alternative  future  [7],  [P01],  [P22].  Solid  methodological 
approaches  to  support  the  implementation  of  this  concept  are  still  in  the  making,  while  it  has  been  reported 
that  in  the  2010  Quadrennial  Defence  Review  the  US  planners  have  used  multiple  sets  of  planning 
scenarios  in  order  to  avoid  optimizing  for  a  single  future  [8].  It  is  possible  to  witness  a  similar  impact  of 
the  Multiple  Futures  Project  [9]  and  related  developments  on  the  NATO  Defence  Planning  Process. 

There  is  a  continuing  tension  between  the  scope  of  the  scenario  set,  i.e.,  the  need  to  span  the  problem 
space,  and  the  constraints  on  time  and  effort  analysts  can  afford  to  spend  on  the  design  and  the  analysis  of 
scenarios.  In  addition,  significant  efforts  are  required  to  maintain  currency  of  scenario  data  and 
assumptions,  as  well  as  to  trace  the  potential  impacts  or  consequences  of  policy  options  and/or  changing 
policy  on  the  scenario  set  used  in  the  defence  planning  process.  Therefore,  only  large  national  defence 
establishments  and  NATO  have  the  capacity  to  analyse  several  dozens  of  planning  scenarios. 

The  identification  of  a  minimum  but  sufficient  scenario  set  is  still  a  challenge.  The  main  issue  in  this 
respect  is  how  to  span  the  envelope  of  roles  and  missions  of  the  armed  forces  without  making  the  costs  of 
analysis  prohibitive.  Formal  analytical  techniques  to  assess  the  coverage  of  the  scenario  set  are  under 
development,  along  with  some  innovations  like  the  use  of  'stressors’  in  analysing  deviations  from  point 
scenarios  with  significant  impact  on  capability  requirements.  Another  enduring  challenge  is  the  practical 
incorporation  of  low  probability  events  with  potentially  catastrophic  effects  in  the  planning  process.  As  a 
consequence,  nations  that  endeavour  to  do  capability-based  planning,  but  have  limited  OA  capacity  may 
explore  fully  only  a  few  scenarios,  and  rely  on  NATO  and  allies  for  the  design  and  the  analysis  of 
common  scenarios. 

3.2  Derivation  of  Capability  Requirements 

As  yet,  there  is  no  generally  agreed  definition  of  capability.  Nevertheless,  practically  all  definitions  refer 
to  capability  as  the  ability  to  solve  an  assigned  task  under  specified  conditions.  The  planning  scenarios 
describe  the  respective  conditions  and  also  serve  to  derive  tasks  to  be  performed  within  the  assigned 
mission  or  for  the  implementation  of  a  specified  strategy,  and  then  to  identify  and  quantify  the  capabilities 
required  to  solve  the  tasks. 

Hence,  analysts  are  required  first  to  decompose  the  assigned  mission  to  tasks,  or — in  other  words — to  do 
“mission-to-task  decomposition.”  The  level  of  formalization  in  this  respect  varies  across  nations  [P09]. 
While  analysts  have  devised  tools  to  assist  this  decomposition,  it  still  requires  solid  military  understanding 
and  judgement.  The  military  officer  decomposes  the  mission  or  the  strategy  with  an  explicit  or  implicit 
concept  of  operations  in  mind.  Effects-based  operations  and  network-centric  warfare  have  been  the 
prominent  concepts  of  operations  in  the  current  wave  of  transformation  [10].  At  this  point,  their 
incorporation  in  the  capability-planning  process  has  not  been  fully  formalized.  Section  5  of  this  report 
provides  more  detailed  examination  of  advanced  concepts  of  operations. 

Capability-based  planning  is  a  term  that  is  already  soundly  established.  Nevertheless,  reflecting  on  the 
impact  of  novel  concepts  of  operations  on  the  planning  process,  the  type  of  defence  planning  currently 
practiced  is  more  accurately  described  as  policy-driven,  concept-led,  scenario-based,  capabilities-oriented 
planning. 
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In  order  to  formalize  the  description  of  capabilities,  the  nations  that  practice  capability-based  planning 
have  designed  their  own  capability  partitions,  or  taxonomies.  Two  broad  approaches  have  been  used  to 
structure  the  defence  capabilities: 

•  Partitioning  along  operational  environments; 

•  Partitioning  along  broad  military  functions. 

Attempts  have  been  made  also  to  structure  defence  capabilities  along  supporting  technologies  and 
systems.  That  approach,  however,  is  considered  ineffective  by  default,  since  it  runs  counter  to  the 
foundations  of  CBP,  i.e.  the  clear  distinction  between  defining  capability  requirements  first  and  the 
follow-on  identification  of  organisational  and  material  solutions. 

The  first  of  these  approaches  better  fits  the  experience  and  the  anticipation  of  military  planners.  It  allows 
using  existing  planning  instruments,  such  as  “universal  task  lists”  and,  through  the  task  lists  provides  for 
smooth  interaction  between  defence  planning,  operations  planning,  and  training.  For  example,  the  U.S. 
defence  establishment  uses  joint,  army,  air  and  naval  task  lists  at  four  levels  -  strategic  national,  strategic 
theatre,  operational,  and  tactical  [11].  Other  countries,  such  as  Canada  and  Australia,  use  three  levels  of 
joint  tasks — strategic,  operational,  and  tactical — and  the  tasks  within  each  level  are  further  disaggregated 
into  two  additional  layers  of  sub-tasks,  with  each  layer  becoming  more  detailed  and  specific  [12]. 

Along  with  its  advantages,  this  approach  has  two  main  drawbacks.  First,  partitioning — and  respective 
capability  requirements — along  operational  environments  almost  directly  translates  to  organizational 
structures,  i.e.  Land  Forces,  Air  Force  and  Navy,  and  thus  it  does  not  directly  contribute  to  the  resolution 
of  the  persisting  problems  of  organisational  stovepipes,  the  related  sub-optimal,  service  specific  capability 
solutions,  and  the  potential  interoperability  problems.  Secondly,  the  maintenance  of  coherent  sets  of  task 
lists,  capability  partitions  and  other  related  methodological  tools,  as  well  as  the  re-use  of  various  products 
of  the  planning  process,  may  be  very  challenging  even  for  nations  with  mature  defence  planning 
arrangements  and  solid  analytical  capacities.4  Countries  that  attempt  to  introduce  capability -based 
planning  through  replication  of  this  approach  are  often  overwhelmed  by  its  analytical  and  organizational 
requirements.  An  attempt  to  deal  with  these  challenges  through  a  less  granular  capability  taxonomy  was 
presented  at  the  symposium  [P  01].  It  utilises  a  single  structured  list  of  capabilities  (not  a  hierarchy)  that 
can  be  visualised  at  different  levels  of  detail. 

The  latter  of  the  two  approaches  is  used  by  the  defence  planning  communities  of  Canada,  The 
Netherlands,  and  seemingly  France.  We  are  not  aware  of  comparative  studies  of  the  effectiveness  and  the 
efficiency  that  can  be  attributed  to  one  or  another  partition.  Both  approaches  may  benefit  from  using 
descriptions  of  capability  categories  that  are  easily  translated  into  effects  that  a  capability  can  deliver. 

All  nations  implementing  CBP  then  conduct  quantitative  capability  analysis  to  derive  requirements  from 
the  planning  scenarios.  A  range  of  operational  analysis  techniques  are  applied  to  enable  quantification. 
That  includes  use  of  simulation  modelling,  linear  programming,  spreadsheets  and  other  purpose-built 
tools,  while  military  expertise  and  judgement  continue  to  be  recognised  as  a  vital  component  of  the 
analysis  [P  09]. 

The  service  oriented  approach  is  particularly  powerful  in  capturing  command  and  control  (C2)  and 
intelligence,  surveillance,  and  reconnaissance  (ISR)  capability  requirements.  It  utilises  a  definition  of 
architectures,  describing  “how  the  elements  within  a  system/organization  interrelate  and  interact  to 
perform  tasks  and  hence  achieve  required  objectives  and  effects”  [PI 4].  It  further  provides  a  systematic 
way  to  derive  a  comprehensive  set  of  requirements,  can  be  scaled  and  adjusted  to  the  problem  at  hand.  It  is 
important  to  underline  that,  once  designed,  architectural  models  can  be  used  across  planning  and  analytical 
communities  for  variety  of  purposes. 

4  For  example,  the  2002  U.S.  Universal  Joint  Task  List  is  a  784-page  document. 
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3.3  Meeting  Capability  Requirements 

The  design  of  solutions  to  capability  requirements  involves  coordination  of  organisational,  doctrinal, 
human,  materiel,  and  information  resources,  individual  and  collective  training.  Nations  use  different 
models  to  describe  capabilities,  e.g.  the  Australian  ‘Fundamental  Inputs  to  Capability’  (FIC)  [13],  the 
Canadian  PRICIE  [P  10],  the  U.K.  TEPID  OIL  [P  02],  etc.  This  difference  is  more  one  of  traditions  than 
of  substance.  The  DOTMLPF  model,  used  initially  by  the  U.S.  military  and  then — with  the  addition  of  T 
for  Interoperability — adopted  by  NATO  as  DOTMLPFI,5  seems  to  be  adequate  in  capturing  all  possible 
ingredients  of  a  defence  capability. 

Hence,  the  issue  is  not  one  of  standardising  capability  models,  but  rather  one  of  paying  due  attention  to  all 
components  of  defence  capabilities,  and  not  focusing  on  personnel  numbers  and  weapon  platforms,  as  it 
happens  just  too  often.  The  report  by  Phil  Guy  [P  16],  for  example,  shows  how  important  is  the  impact  of 
guided  munitions  stockpiles  both  on  costs  and  on  capability  levels.  Guy  shows  that  the  use  of  robust 
numerical  methods,  combined  with  scenario  analysis,  is  the  most  suitable  approach  to  quantitative 
planning  of  strategic  munitions  stockpiles  within  the  overarching  capability-based  planning  framework. 
He  reasons  that  a  strategic  munitions  planning  model  has  to  account  for  existing  stockpiles,  munition 
costs,  platform  survivability,  minimum  deployable  quantity,  target  interdependence,  target  range,  platform 
availability  and  time  constraints,  inter-service  conflicts,  false  targets  (the  limited  ability  to  identify  targets 
correctly,  either  because  of  skilful  deception  or  bad  weather),  other  enemy  actions,  losses  onboard  lost 
platforms,  the  eventual  need  to  expend  some  munitions  to  ‘zero-in’  on  targets,  logistics  allowance, 
technical  failure,  environmental  factors  such  as  weather,  terrain,  temperature,  humidity,  dust,  etc.,  non- 
doctrinal  usage,  and  geographical  location.  Guy  presents  analysis  of  recent  conflict  that  generally  confirms 
the  need  to  consider  all  these  factors,  telling  us  at  the  same  time  that  against  an  ill-defined,  militant,  non- 
conventional,  regenerating  opposing  force  it  is  more  challenging  to  define  both  target  lists  and  criterion  for 
its  defeat.  Nevertheless,  the  author  still  follows  a  primarily  target-oriented  methodology.  Analysing 
results,  he  formulates  policy  recommendations,  related  to  standardization  and  multi-national  procurement 
of  strategic  munitions. 

The  “capability  portfolio”  is  emerging  as  a  dominant  concept  in  the  search  of  answers  to  the  defence 
planning  task.  Implementing  the  concept  of  a  portfolio,  defence  planners  seek  comprehensive  solutions, 
examining  all  types  of  required  capabilities  at  one  point  in  the  planning  process.  Planners  identify  gaps 
and  redundancies  in  the  current  set  of  capabilities,  monitor  and  analyse  emerging  technologies  and  their 
impact,  design  capability  options  and  seek  synergies. 

Rigorous  prioritization  of  capability  requirements  or  of  capability  gaps  further  assists  the  identification  of 
solutions  that  decision  makers  would  find  acceptable.  A  methodology,  presented  at  the  symposium  [P  15], 
prioritises  capability  gaps  based  on  their  “relative  importance”  calculated  as  an  average  of  their  weight, 
added  utility,  and  urgency.  In  the  calculation  of  the  weight  analysts  process  expert  judgements  on  the 
relative  importance  of  one  scenario  vis  a  vis  other  scenarios  in  the  set  of  planning  scenarios,  using  a  nine 
degree  Saaty  scale.6  Then  they  calculate  coefficients  of  ‘participation’  (based  on  the  number  of  scenarios 
which  require  the  respective  capability),  of  ‘risk’  (the  potential  impact  on  operational  objectives  if  the 
capability  requirement  is  not  met),  and  of  ‘correlation’  (to  account  for  the  mutual  influence  among 
capabilities  in  an  operation).  At  the  next  steps  analysts  use  an  exponential  utility  function  in  processing 
military  assessments  of  the  potential  increase  of  capability  levels.  The  authors  provide  an  example  and  end 
with  a  claim  that,  compared  to  the  ACT  methodology  [15],  this  one  adds  content  and  clarity  to  a 
commander’s  strategic  vision  and  enhances  the  quality  of  information  available  to  decision  makers. 


5  Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel,  Facilities,  and  Interoperability. 

6  That  approach  differs  from  the  one  in  the  2004  CBP  Guide  [14]  where  the  selected  set  includes  plausible  scenarios,  and  no 
distinction  of  their  relative  importance  is  made. 
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In  this  and  other  studies  of  prioritization,  the  definition  of  priorities  includes — either  implicitly  or 
explicitly — assessment  and  understanding  of  planning  risks.  By  its  design,  the  portfolio  approach  makes 
assessment  of  risks  explicit.  Practically,  in  defining  the  capability  portfolio,  planners  attempt  to 
minimize — with  different  degree  of  rigour — some  integral  criterion  of  planning  risks.7  Thus,  the  portfolio 
approach  directly  supports  the  institutionalization  of  risk  management  and  related  frameworks  in  the 
defence  planning  process. 

The  presentations  and  the  discussions  during  the  symposium  outline  a  trend  towards  increased  coverage  of 
the  capability  portfolio  planners  are  trying  to  come  up  with.  First,  most  planners  already  include  in  the 
portfolio  various  support  capabilities.  In  Canada,  for  example,  capabilities  of  supporting  defence 
organisations  are  included  in  the  portfolio  through  the  “Generate”  domain  [P  10].  In  a  second  example, 
Bulgarian  analysts  envision  further  expansion  of  the  capability  portfolio  in  an  attempt  to  incorporate 
efficiently  national  defence  strategies,  to  capture  the  full  costs  of  defence  capabilities  and  to  be  able  to 
match  it  to  anticipated  budget  levels.  Thus,  it  was  suggested  to  add  four  domains  to  the  traditional  six 
“operational  capabilities.”8  These  four  domains  are  [P  01]: 

•  Generating  capabilities  and  maintaining  readiness; 

•  Strategic  management; 

•  Shaping  regional  security  environment;  and 

•  Strategic  adaptability  and  organizational  agility. 

A  third  direction  of  expansion  of  the  capability  portfolio  reflects  the  comprehensive  approach  to 
operations  and  supports  the  implementation  of  concepts  like  3D  (Defence,  Diplomacy,  Development  aid) 
and  Whole-of-Government  approach  to  operations.  A  variety  of  organisations  and  not  just  the  military 
develop  and  provide  requisite  capabilities  to  operations  such  as  the  ISAF  operation.  Respectively,  the 
planning  methodology  needs  to  be  adapted,  e.g.  through  expansion  of  capability  portfolios  under 
examination,  or  adapting  the  capability  taxonomy. 

A  variation  of  that  expansion  relates  to  the  extension  of  CBP  to  other  security  domains.  For  example,  the 
U.S.  Department  of  Homeland  Security  (DHS)  implements  a  planning  methodology  that  is  almost  a  direct 
reflection  of  the  capabilities -oriented  methodology  used  by  defence  planners.  DHS  uses  a  set  of  planning 
scenarios,  a  universal  task  list,  and  a  target  capabilities  list,  forming  “the  basis  for  coordinated  federal 
planning,  training,  exercises,  and  grant  investments  needed  to  prepare  for  emergencies  of  all  types”  [17]. 

Eventually,  the  CBP  approach  may  be  applied  to  support  holistic,  rigorous  and  transparent  decision 
making  on  the  development  of  national  security  sectors,  e.g.  in  the  implementation  of  concepts  of  security 
sector  reform  or  security  sector  transformation  [P  11].  Attempts  have  already  been  made  to  extend  the 
CBP  methodology  for  that  purpose  [5].  At  least  in  one  case,  in  the  Netherlands,  the  CBP  has  been  applied 
in  a  government-wide  national  security  planning  process  [18].  It  may  turn  out  that  the  challenge  is  not 
primarily  analytical:  the  fact  that  DHS  relies  on  a  rather  straightforward  application  of  the  capability- 
oriented  defence  planning  methodology  indicates  that  CBP  is  in  fact  transferable  to  other  security 
domains.  Most  likely,  the  main  challenges  lie  in  the  need  to  coordinate  distinct  bureaucratic  processes  and 
to  de-conflict  different  organisational  cultures.  The  defence  analytical  community  is  in  position  to  initiate 
a  study  on  the  applicability  and  to  promote  a  holistic  capability-based  planning  framework  for  national 
security  sectors,  thus  contributing  to  the  effectiveness  and  the  efficiency  of  the  response  to  security 
challenges  within  ever  stricter  budgetary  constraints. 


7 

The  2008  defense  strategy  of  the  United  states  calls  these  risks  “Future  challenges  risks”  [16]. 

g 

NATO  and  several  nations  use  variations  of  the  following  six  domains:  Effective  Engagement;  Effective  Intelligence; 
Deployability  and  Mobility;  Command  and  Control;  Logistics  Support;  Survivability  and  Force  Protection. 
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Once  a  realistic  portfolio  of  defence  capabilities  has  been  defined,  some  nations  turn  to  balance  of 
investments  studies  [19],  thus  providing  a  clear  linkage  between  capability  requirements  and  investment 
decisions,  with  the  respective  translation  of  priorities. 

3.4  The  Search  for  End-to-End  Solutions 

Several  presentations  and  the  following  discussion  during  the  symposium  provide  grounds  to  identify 
another  trend  -  the  search  for  ‘end-to-end’  solutions  in  capability-based  planning.  J-DARTS  9  is  seen  as  a 
promising  tool  to  provide  such  solution  on  a  national  level  [P  13].  Norway  is  ahead  among  NATO  nations 
in  adapting  J-DARTS  to  provide  efficient  support  to  capability -based  planning  for  the  needs  of  the 
country’s  defence  establishment. 

In  order  to  present  and  elaborate  on  plausible  execution  of  defence  missions,  Norwegian  planners  and 
analysts  use  J-DARTS  to  develop  concrete  situations  with  defined  geography,  actors  and  time  lines.  These 
situations,  or  scenarios,  are  then  used  to  decompose  each  mission  type  into  objectives,  tasks  and  sub-tasks, 
and  to  determine  the  type  and  level  of  capabilities  required  for  each  sub-task.  Further,  Joint  Activity  Trees 
and  respective  ‘Capability  Assignment  Logic’  are  used  to  quantify  generic,  potential  solutions  to  ‘Key 
Tasks’.  All  capability  requirements  from  all  scenarios  are  aggregated  into  so  called  ‘benchmarks’, 
corresponding  to  specified  levels  of  ambition.  A  Mixed  Integer  Programme  employing  the  CPLEX 
optimization  algorithm  is  used  to  match  forces  to  benchmarks  and  scenarios  and  to  suggest  a  future  force 
structure  at  the  lowest  possible  cost.  In  addition,  J-DARTS  serves  as  repository  of  data  and  analysis 
results,  thus  allowing  re-use  and  increase  in  efficiency  in  follow-up  planning  cycles. 

The  tendency  to  search  for  and  implement  end-to-end  analytical  support  tools  may  be  strengthened  with 
the  use  of  J-DARTS  in  support  of  the  new  NATO  defence  planning  process,  as  well  as  by  attempts  of 
other  NATO  nations  to  use  J-DARTS  to  guarantee  the  transparency  of  allied  planning  or  for  their  national 
defence  planning  purposes,  or  both. 

While  initially  used  by  Norway  to  support  defence  reviews,  conducted  in  a  four-year  cycle,10  J-DARTS  is 
currently  seen  as  “one  of  the  main  tools  underpinning  the  new  continuous  planning  process  ... 
implemented  by  the  Norwegian  MoD”  [P  13].  Mr.  Mark  Rempel  from  the  Centre  for  Operational  Research 
and  Analysis  -  Defence  Research  and  Development  Canada  (DRDC)  presented  another  example  of 
comprehensive  analytical  support  to  a  continuous  capability  development  process  [P  10]  in  support  of  the 
finding  that  “some  nations  are  transitioning  from  a  cyclical  to  a  continuous  model  for  their  defence 
planning”  [P  09]. 

In  the  second  generation  strategic  decision-making  process  for  the  Department  of  National  Defence 
(DND)  and  the  Canadian  Forces  (CF),  the  capability-based  planning  is  an  integral  part  of  the  Canadian 
force  development  process,  which  in  turn  is  incorporated  in  the  CF/DND  strategic  decision-making 
governance  structure  [P  10],  [20].  Planners  combine  in  one  phase  of  the  planning  process  “capability 
planning,  management,  and  integration”  that,  in  combination,  produce  two  key  products:  the  strategic 
capability  roadmap  (or  “Defence  Capability  Plan”)  and  the  investment.  Through  two  key  feedback 
loops — between  the  ‘Capability  Based  Planning’  and  ‘Strategic  Guidance’  components  and  between  the 
‘Key  Products’  and  ‘Capability  Based  Planning’  components — planers  continuously  update  the  key 
planning  products,  thus  keeping  the  force  development  process  adequate  to  the  evolving  force 
development  environment. 


9  The  Joint  Defence  Planning  Analysis  and  Requirements  Tool  Set  (J-DARTS)  has  been  continuously  developed  and  enhanced 
by  the  NATO  Consultation,  Command  and  Control  Agency  (NC3A)  since  2001.  It  has  been  provided  to  several  NATO 
nations,  willing  to  use  it  for  national  planning  purposes. 

10  And  the  2007  Defence  Study  in  particular. 
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To  support  the  planning  process,  DRDC  has  developed  and  supports  the  implementation  of  five  analytical 
methods  and  two  subject  matter  expert  analyses  (risk  outlook  and  alternative  to  project  mapping)  [P  10]. 
The  five  methods  are: 

•  The  CATCAM  method  is  used  to  evaluate  and  prioritize  the  capability  framework  elements  within 
force  planning  scenarios  through  subject  matter  experts  assessing  each  of  six  standardized  mission 
effects  with  respect  to  their  required  frequency  and  the  consequence  if  the  effect  is  not  created 
within  each  scenario;  it  produces  a  numeric  score  for  each  capability  element  across  effects; 

•  The  Scenario  Capability/Capacity  Requirements  Assessment  and  Outlook  Tool  (SC2RAT)  is  used 
to  process  capability  element  scores,  existing  and  future  programmed  operational  force  structure, 
and  force  element  rotation  ratios  and  to  generate  a  capability  outlook  that  effectively  describes  the 
health  of  the  CF  capabilities  over  time  for  each  force  planning  scenario; 

•  The  Concurrency  method  is  used  to  investigate  the  force  element  requirements  of  concurrent 
scenarios; 

•  The  Optimization  component  of  the  CBP  analytical  process  is  used  in  the  exploration  of  the 
solution  space  for  non-dominated  sets  of  solutions  that  best  address  the  identified  capability 
deficiencies;  a  solution  is  comprised  of  a  set  of  capability  investment  alternatives; 

•  The  Cost  Sensitivity  method  is  applied  to  investigate  the  risk  of  delivering  alternative  solutions 
due  to  cost  risk,  assuming  a  triangular  distribution  of  equivalent  annual  costs  for  each  alternative. 

The  work  of  the  analysts  from  Defence  Research  and  Development  Canada  provides  one  example  of  a 
planning  and  force  development  process  that  is  getting  away  from  the  comfortable  regularity  of  review 
and  planning  cycles  with  clear  cut  delineation  of  long-,  mid-  and  short  term  planning  horizons.  Several 
other  countries  are  also  transitioning  to  a  process  of  continuously  seeking  to  balance  security 
requirements,  defence  objectives,  capability  development  plans,  and  affordability  considerations.  To  this 
purpose  policy  makers  and  planners,  supported  by  analysts,  introduce  a  feedback  loop  from  the  assessment 
of  performance  and  results,  i.e.  the  level  of  capabilities  achieved  and  the  efficiency  in  utilising  limited 
resources.  Lately,  this  continuous  cyclic  process  has  been  referred  to  as  “strategic  defence  management” 
[19],  [P  01]. 

3.5  Credibility  of  Defence  Planning  and  Plans 

Notwithstanding  the  level  of  analytical  effort  and  rigour  in  designing  force  structures  and  proposing  force 
development  plans,  investment  plans,  recruitment  targets,  etc.,  no  analyst  would  suggest  that  defence  plans 
represent  an  objective,  scientific  Truth’  [P  13].  Nevertheless,  analysts  are  expected  to  provide  a 
transparent,  auditable  decision-making  process  that  thoroughly  explores  solution  spaces  and  allows  to  deal 
with  uncertainty. 

First,  the  rationale  for  decisions  has  to  flow  transparently  through  all  phases  of  the  decision  making 
process.  A  decision  maker  would  most  probably  not  be  able  to  follow  all  details,  but  needs  to  maintain  a 
strategic  level  view  throughout  the  process.  DRDC  relates  to  this  analytical  support  challenge  as 
“enhancing  the  Command  View”  and  lists  the  design  and  development  of  an  adequate  interface  between 
senior  decision-makers  and  the  CBP  process  among  the  main  development  thrusts  for  the  next  generation 
analytical  support  [P  10]. 

Second,  with  a  certain  increase  of  the  analytical  effort,  planners  can  explore  both  the  problem  and  the 
solution  spaces  more  systematically.  Further  methodological  efforts  are  necessary  to  guarantee  that  the  set 
of  planning  scenarios  used  in  CBP  span  the  space  of  relevant  security  challenges  reasonably  well  (from 
the  point  of  view  of  decision  makers),  without  making  the  analysis  cost  prohibitive.  Likewise,  the  solution 
space  needs  to  be  fully  explored,  with  account  for  capabilities  and  their  respective  organizational  and 
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material  ‘providers’  to  be  maintained  and  those  to  be  divested.  While  traditional  optimization  methods  and 
tools  preserve  their  value,  recent  advances  in  the  field  of  genetic  algorithms  also  prove  beneficial  in 
increasing  the  planers’  confidence  in  the  quality  of  the  solutions  [P  10]. 

Third,  even  best  efforts  in  designing  and  selecting  the  set  planning  scenarios  represent  the  planners’ 
understanding  of  the  future  world  and  may  prove  in  error  in  the  face  of  deeper  uncertainty.  According  to 
Colin  S.  Gray,  a  contemporary  strategist,  “most  efforts  to  anticipate  the  future  have  been  seriously  in  error. 
There  has  usually  been  somebody  who  got  it  right  enough  at  the  time,  but  at  the  time  people  had  no  way  of 
knowing  who  that  person  was”  [21].  Hence,  one  seemingly  promising  way  to  represent  such  uncertainty  is 
not  to  anticipate  a  single  vision  of  the  world,  but  to  envision  multiple  futures,  each  of  which  can  be 
represented  in  the  planning  process  with  its  own  set  of  scenarios  11  [9].  Attempts  to  formalize  the  concept 
for  defence  planning  purposes  have  already  been  made  [7],  [P  01],  but  so  far  the  issue  of  traceability  has 
not  been  rigorously  addressed. 

Fourth,  planners  cannot  provide  credible  solutions  if  the  data  they  use  is  not  reliable.  One  apparent  area  of 
concern  is  costing.  Reliability  of  capability  costing  can  be  increased  with  the  use  of  proven  cost  models 
[22],  including  models  of  life  cycle  costs  [23],  systematic  accumulation  and  validation  of  cost  data.  Both 
stochastic  and  deterministic  models  of  assessing  future  costs  of  capability  alternatives  have  been  used  in 
capability-based  planning  [P  10].  Benchmarking  studies  of  both  models  and  costs  can  further  enhance  the 
confidence  in  national  approaches,  as  well  as  the  understanding  among  allies  of  the  costing  aspects  of 
defence  planning. 

The  analysis  of  sensitivity  of  planning  solutions  to  main  assumptions  and  human  estimates,  albeit  heavy  in 
terms  of  workload,  also  contributes  to  the  credibility  of  the  analytical  work. 

Finally,  planners  and  analysts  are  expected  to  maintain  an  audit  trail  that  can  be  used  to  trace  the  flow  of 
the  decision  making  process  from  the  guidance  to  decisions  on  capabilities,  force  structure,  and  investment 
projects.  The  analytical  community  should  avoid  trend  extrapolation  and  wishful  thinking  and  try  to  keep 
alive  a  long-term  planning  and  study  process,  including  alternatives,  adding  to  that  critical  assessment  of 
today’s  capabilities  and  assessments  by  simulation  and  gaming  [P  04]. 

3.6  On  the  Efficiency  of  the  Defence  Planning  Effort 

Capability-based  planning  in  its  already  traditional  form  [14]  is  highly  analytically  demanding,  in 
particular  when  a  higher  number  of  planning  scenarios  is  included  in  analysis.  The  demand  for  analytical 
support  increases  radically  with  the  transition  to  a  continuous  planning  model,  the  introduction  of  rigorous 
sensitivity  analysis  and  the  representation  of  deeper  uncertainty  through  multiple  sets  of  planning 
scenarios. 

The  increase  of  efficiency  of  analytical  support  is  expected  to  parallel  the  increase  of  analytical  demand  if 
we  do  not  want  to  see  the  cost  of  analytical  support  escalating.  One  approach  to  increase  efficiency  is  to 
seek  compatibility  and  higher  degrees  of  standardization  of  capability  partitions,  models,  generic  units, 
etc.,  used  by  defence  analysts  of  NATO  and  NATO  nations.  That  will  have  important  side  effects  in 
providing  compatibility  of  national  defence  planning  process  and  increasing  the  transparency  of  NDPP. 
The  RTO  SAS  panel  has  and  will  continue  to  play  an  important  role  in  that  respect  through  sharing  of 
information,  experience,  and  benchmarking  studies. 

A  complementary  trend  involves  implementation  of  advanced  IT  systems  and  tools.  Advanced  geographic 
information  systems,  database  and  content  management  systems  increase  the  efficiency  of  planning  in 
variety  of  ways,  e.g.  allowing  for  re-use  of  scenarios  designed  for  different  purposes  [P  13],  re-use  of 


11  Obviously,  there  will  be  an  overlap  among  the  sets  of  planning  scenarios. 
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analysis  results,  etc.  A  drive  to  implement  a  CBP  knowledge  management  system  is  also  expected  to 
contribute  to  the  efficiency  of  analytical  support  [P  10]. 


4.0  ANALYTICAL  SUPPORT  TO  THE  CAPABILITY  DEVELOPMENT 
PROCESS 

Effective  and  efficient  capability  delivery  is  based  on  coordinated  development  of  human,  material, 
information  components,  training  and  interoperability,  described  in  NATO  by  the  DOTMLPFI  model.  Of 
these,  the  symposium  discussed  the  analytical  challenges  of  acquisition  and  personnel  management  and 
touched  on  logistics  issues. 

4.1  Analytical  Support  to  Personnel  Management 

Capabilities  cannot  be  delivered  without  adequately  educated,  trained  and  motivated  people.  And  yet,  the 
number  of  such  people,  willing  to  serve  in  the  armed  forces,  is  subject  to  policy  decisions,  often  having 
long-term  impact  on  the  process  of  recruitment  and  retention.  Analysts  can  contribute  to  the  understanding 
of  trends  and  drivers,  as  well  as  to  the  definition  of  “capability-oriented”  personnel  policy  and 
programmes.  The  study  on  personnel  attrition  rates  by  Manchun  Fang  and  Paul  Bender  from  the 
Department  of  National  Defence  of  Canada,  Director  General  Military  Personnel  Research  and  Analysis, 
is  informative  in  that  respect  [P  17].  Fang  and  Bender  present  a  model  of  quantifying  demographic  and 
behaviour  effects  on  changes  in  attrition  rate.  The  model  is  applied  to  identify  policy  changes — in  the 
process  of  personnel  selection,  remuneration,  and  others — that  have  caused  particular  effects  on  attrition 
rates.  The  lessons  from  the  study,  as  well  as  the  developed  model,  can  be  used  to  support  future 
development  of  targeted  retention  initiatives. 

4.2  Analytical  Support  to  Acquisition  Management  and  Logistics 

Thomas  Ekstrom  from  FOI  took  the  most  comprehensive  view  to  the  ongoing  transformation  of  defence 
procurement  and  logistics  [P  19].  Driven  by  the  post-Cold  war  changes  in  security  requirements,  the 
opportunities  created  by  new  information  and  communications  technologies  and  the  impact  of  new 
business  concepts,  the  Revolution  in  Military  Logistics  (RMF)  paralleled  the  Revolution  in  Military 
Affairs  (RMA),  and  now  the  process  of  defence  transformation.  A  previously  rather  straightforward 
military  supply  chain  turned  into  an  increasingly  complex  and  fragmented  supply  and  support  network, 
with  many  actors  and  roles  and  responsibilities  that  are  not  entirely  clear.  Respectively,  during  the  Cold 
war  analysts  focused  on  scenario  techniques,  war-gaming,  modelling  and  simulation,  cost-effect  or  cost 
benefit  analysis,  and  problem  structuring  methods,  such  as  morphological  analysis.  Mr.  Ekstrom  identifies 
a  number  of  areas  currently  in  need  of  analytical  studies  and  support,  such  as: 

•  Design  and  management  of  supply  and  support  chains  and  networks; 

•  Supply  and  support  chain  risk  management; 

•  Identification  and  evaluation  of  potential  business  models  for  defence  acquisition  and  logistics  in 
the  entire  spectrum  from  public  provision  to  outright  privatization; 

•  Development  of  Decision  Support  Systems  (DSS),  decision  trees,  decision  mechanisms,  and  other 
support  to  decisions  regarding  what  to  make,  what  to  buy,  and  how  to  buy; 

•  Evaluation  of  efficiency,  effectiveness  and  performance,  including  definition  of  measures  of 
effectiveness  (MOEs)  and  measures  of  performance  (MOPs); 

•  Moral  and  ethical  issues  associated  with  outsourcing,  in  particular  in  hostile  environment; 

•  Defence  acquisition  culture,  organisation,  and  competencies. 
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Wlodzimierz  Miszalski  and  Szymon  Mitkow  from  the  Military  University  of  Technology  in  Warsaw 
presented  an  original,  comprehensive  approach  to  support  the  process  of  defence  acquisition,  starting  from 
the  definition  of  capability  requirements  through  selection  of  a  material  solution  all  the  way  to 
optimization  of  the  respective  logistics  support  [P  18].  The  authors  use  the  AHP  (Analytic  Hierarchal 
Process)  method  in  solving  the  multicriteria  problem  of  selecting  a  capability  solution  among  a  number  of 
alternatives.  The  following  criteria  are  used  by  decision  makers:  combat  ability,  cost,  technological 
feasibility,  time  (time  required  for  delivery,  time  in  service),  availability  on  the  defence  equipment  market, 
logistics  requirements,  and  versatility  (is  the  capability  applicable  across  tasks).  Once  the  class  of  a 
material  solution  has  been  defined,  analysts  turn  to  the  selection  of  a  product  from  that  class.  The  authors 
create  a  benchmark — an  ‘ideal’  solution  having  the  best  features  among  the  features  of  interest  of  all 
available  products — and  then  rank  the  products  according  to  the  distance  from  the  benchmark.12  Thus, 
while  admitting  that  political  and  economic  considerations  often  come  into  play,  the  analysts  put  the 
product  that  is  closest  to  the  benchmark  on  top  of  the  list  of  possible  material  solutions. 

Robert  Murphy  and  Scott  D.  Beach  from  Lockheed  Martin  Aeronautics  presented  another  comprehensive 
decision-making  framework,  setting  procurement  decisions  within  the  strategic  planning  process,  i.e.  the 
top-down  process  of  developing  scenarios,  defining,  developing  and  deploying  a  strategy  [P  20]. 
Identification  of  metrics  is  part  of  the  step  of  developing  the  strategy.  Identification  of  capability  solutions 
and  analysis  of  alternatives,  including  effectiveness,  cost  and  risk  analysis,  are  all  part  of  the  “deploy  the 
strategy”  step.  The  Force  Matrix  Model™  (FMM)  developed  by  Lockheed  Martin  supports  rigorous 
analytical  studies  of  the  whole  process,  including  evaluation  of  effectiveness,  cost,  and  risk  of  identified 
solutions  by  using  multi-attribute  value  theory  (MAVT)  which  includes  Analytical  Hierarchy  Process 
(AHP)  and  utility  curves.  Seen  as  an  alternative  to  expensive  and  time  consuming  modelling  and 
simulation,  MAVT  uses  [P  20] : 

•  Criteria  or  Measures  of  Performance  developed  for  each  capability  and  then  prioritised  using  the 
AHP  process; 

•  An  AHP  process  that  uses  a  pair-wise  comparison  of  capabilities  and  then  ranks  them  using  an 
arithmetic  or  geometric  mean; 

•  Utility  curves  for  determining  the  value  of  a  solution;  using  s-curves,  FMM  mimics  the  way 
people  make  decisions  and  most  often  represent  this  thought  process  by  allowing  the  decision 
maker  to  define  the  “preference”  curve. 

The  final  result  is  a  weighted  average  (or  normalized  ranking)  of  the  capabilities  rolled  up  into  a  single 
overall  value. 


5.0  ANALYTICAL  SUPPORT  TO  CONCEPT  DEVELOPMENT  AND 
EXPERIMENTATION  (CD&E) 

The  CD&E  process  affirms  its  role  in  the  capability  development  process  of  NATO,  NATO  nations  and 
partner  countries.  Capability  development  is  understood  to  encompass  “strategic  analysis,  identification  of 
capability  requirements,  solution  identification  and  solution  implementation”,  while  CD&E  enables 
“structured  development  of  creative  and  innovative  ideas  into  viable  solutions”  [P  21].  That  does  not  come 
quickly.  CD&E,  and  innovation  in  general,  involves  not  only  technologies  and  processes,  but  also  the 
people  in  the  organisation.  A  process  of  continuous  learning  accelerates  innovation,  as  eloquently 
summarised  by  John  J.  Garstka: 

One  factor  complicating  military  transformation  is  the  need  to  demonstrate  the  potential  operational 
effectiveness  of  a  new  concept.  In  the  case  of  inter-war  carrier  aviation  and  mechanised  warfare,  the 
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Including  in  the  ‘distance’  also  a  measure  of  the  dispersion  of  the  features. 
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initial  operational  concepts  and  associated  technologies  failed  to  perform  as  well  as  existing 
capabilities.  Few  appreciated  the  potential  impact  that  the  operational  concepts  and  technologies 
would  have  as  they  matured.  In  both  cases,  the  ability  to  conduct  a  mutually  reinforcing  series  of 
experiments,  exercises  and  war-games  was  critical  to  enabling  visionary  leaders  to  accelerate  learning 
rates  and  obtain  evidence  to  support  investing  in  emerging  capabilities.  In  the  case  of  mechanised 
warfare,  a  critical  mass  of  individuals  in  the  German  Army  was  able  to  learn  about  how  mechanised 
forces  could  be  employed  much  faster  than  their  peers  in  the  British,  French  or  Polish  armies  [24] . 

This  section  of  the  report  is  structured  in  three  sub-sections,  reflecting  symposium  discussions  on  recent 
developments  of  the  CD&E  process,  the  proper  understanding  of  “analytical  rigour”  in  experimentation, 
and  the  roles  of  the  analytical  community. 

5.1  Towards  a  Common  Understanding  of  CD&E  and  CD&E  Project  Management 

Recent  developments  within  NATO  have  contributed  to  the  clarification  of  the  role  of  CD&E  and  to  the 
common  understanding  of  the  phases  of  the  CD&E  process,  its  financing,  tracking,  and  quality  assurance. 
In  September  2009,  the  NATO  Military  Committee  has  approved  the  Alliance  policy  for  Concept 
Development  and  Experimentation  [25].  The  symposium  participants  generally  agreed  that  this  NATO 
policy  is  in  line  with  the  “Guide  for  Understanding  and  Implementing  Defense  Experimentation” 
(GUIDEx),  developed  by  the  Joint  Systems  Analysis  Group  in  the  framework  of  The  Technical 
Cooperation  Program,  involving  defence  experimentation  experts  from  Australia,  Canada,  the  United 
Kingdom  and  the  United  States  [26]. 

CD&E  plays  an  important  role  in  finding  conceptual  solutions  to  capability  shortfalls  and  gaps  that  were 
identified  in  the  processes  of  defence  planning,  lessons  learned,  or  as  urgent  operational  requirements.  It 
also  contributes  to  capability  development  through  the  introduction  of  capabilities  not  anticipated 
previously.  They  may  result  from  new  ideas,  “out  of  the  box”  thinking  or  systematic  R&T  efforts  [P  21]. 

CD&E  is  the  process  of  structured  development  of  innovative  ideas  into  viable  solutions.  This  is  an 
iterative  process  leading,  eventually,  to  an  approved  concept.  Concept  development  provides  the 
framework  within  which  a  solution  may  be  developed,  and  solutions  may  (or  may  not)  be  refined  through 
experimentation  [P  21].  Experimentation  reduces  uncertainty  as  to  the  maturity  of  the  concept  or  its 
component  parts. 

Although  CD&E  is  a  concurrent,  iterative  and  spiral  process,  in  which  Concept  Development  and 
Experimentation  are  closely  interlinked,  for  project  management  purposes  it  is  useful  to  distinguish  four 
phases  [P  21]: 

a)  Preparation,  Initiation  and  Structuring; 

b)  Concept  Development  Planning; 

c)  Concept  Development  and  Refinement;  and 

d)  Assessment  and  Validation. 

Each  of  these  phases  poses  specific  challenges  to  analysts,  which  will  be  examined  in  more  detail  below. 

While  the  NATO  CD&E  policy  provides  a  benchmark  for  a  comprehensive,  disciplined  management  of 
CD&E  projects,  it  may  be  beneficial  to  take  a  look  at  the  process  from  a  different  perspective.  Another, 
“civilian”  perspective  from  the  experience  of  the  business  world  was  introduced  by  Dr.  Venelin  Georgiev, 
who  drew  parallels  between  the  CD&E  process  and  the  process  of  innovation.  Sound  innovation 
management  supports  the  decision  making  process  through  exploration  of  the  decision  space  and  the  use 
of  scenarios,  and  reduces  the  associated  technical,  technological,  financial,  and  planning  risks.  Georgiev 
showed  how  that  the  decision  process  becomes  more  rational,  transparent  and  accountable  [P  24]. 
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Another  report  in  the  CD&E  session  addressed  organisational  effectiveness  in  coalition  HQs  conducting 
non-article  5  crisis  response  operations,  presenting  interim  results  of  a  RTO  HFM  study  group  [P  27].  The 
authors  show  that  in  order  to  implement  the  concept  of  NATO  Network  Enabled  Capabilities  (NNEC),  the 
military  organisation  needs  to  be  agile,  flexible,  joint  and  interoperable.  Such  organic  organisational 
structure  has  the  following  attributes:  it  is  decentralised;  the  authority  to  make  important  decisions  is 
delegated  to  persons  at  all  levels  of  the  hierarchy;  it  stimulates  flexibility,  so  that  employees  can  innovate 
and  quickly  adapt  to  changing  circumstances,  and  take  responsibility  to  make  decisions  when  necessary; 
roles  are  loosely  defined;  organisational  members  with  different  functions  work  together  to  solve  problems 
and  are  involved  in  each  other’s  activities;  a  high  level  of  integration  enables  organisational  members  to 
share  information  quickly  and  easily;  rules  and  norms  emerge  from  the  ongoing  interaction  between 
organisational  members;  and  interaction  between  organisational  members  is  horizontal  as  well  as  vertical. 
The  study  group  came  up  with  the  following  basic  characteristics  of  an  effective  Coalition  HQ: 

•  Political-military  decision  making:  Able  to  achieve  its  goals;  Establishing  priorities; 

•  Internal  processes  management:  Learning  organisation;  Stimulating  information  sharing;  The  HQ 
is  willing  to  adapt  its  structures  to  the  ever-changing  conditions  where  necessary;  Processes 
improvement  strategies  implementation  to  facilitate  information  sharing,  social  networking  and 
top  leaders’  commitment  to  achieving  HQ  goals;  Making  efficient  use  of  the  available  resources; 

•  People:  Able  to  take  initiative;  The  leaders  are  able  to  make  fast  and  timely  decisions;  Existing 
flexible  human  resources  management  system  to  guarantee  high  motivation,  cohesion, 
organisational  and  interpersonal  trust; 

•  Cultural  differences:  Openness  to  diverse  cultures;  development  of  intercultural  competences; 
Using  common  language  and  terminology;  Using  common  formats/standardization  of  different 
procedures;  Using  common  doctrine  and  concepts. 

They  also  show  that  for  the  NATO  HQ  to  be  able  to  attain  its  goals  of  effective  and  timely  sharing  of 
information,  quick  and  timely  decision  making,  and  improved  shared  awareness  of  tasks  and 
responsibilities,  its  terminal  cultural  values  must  reflect  flexibility  and  agility  in  its  processes,  but  stability 
in  the  organisational  structure,  and  its  instrumental  cultural  values  should  include  trusting  each  other, 
being  open  to  diversity,  and  having  an  improvement  orientation. 

5.2  How  Rigorous  Can  Be  the  Experiment? 

Describing  Swedish  experience  in  defence  transformation,  Jan  Foghelin  underlined  the  need  to  take  into 
account  new  ideas  and  concepts.  At  the  same  time,  he  emphasised  that  even  if  new  concepts  “are 
immediately  seducing,  they  should  not  be  a  substitute  for  studies  -  should  be  transferred  to  tangible 
solutions  and  then  tested  in  several  dimensions  ...”  [P  04].  In  his  presentation  in  the  CD&E  session,  Han 
de  Nijs  also  supported  this  point,  stating  that  analysis  activities  along  the  CD&E  process  are  necessary  to 
accumulate  evidence  in  order  to  determine  and  demonstrate  the  validity  of  proposed  solutions  and  the 
increase  of  effectiveness.  Analytical  rigour  is  important  to  increase  confidence  in  a  conceptual  solution 
and  reduce  the  risk  involved  with  its  implementation  [P  21]. 

But  how  rigorous  can  be  an  experiment  in  the  context  of  defence  transformation? 

There  is  no  simple  answer  to  this  question.  One  needs  to  delve  into  the  foundation  of  science  to  find  an 
adequate  answer.  Generations  of  scientists,  in  particular  in  the  fields  of  natural  sciences  and  engineering, 
have  been  educated  in  the  paradigm  of  “positivism”,  where  “theory  corresponds  to  the  only  one,  true, 
objective  reality”  [P  25].  In  studying  that  reality,  e.g.  through  experimentation,  analysts  attempt  “to  be 
objective,”  i.e.  to  “ensure  that  no  bias,  subjective  influences  or  value  considerations  disrupt  our  view  of 
the  true  reality”  [P  25]. 
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Andy  Williams  and  Simon  Purton  show  that  while  this  scientific  method  is  appropriate  in  many  cases  of 
defence  experimentation,  it  may  be  counterproductive  in  experiments  that  bring  together  political 
sensitivities,  divergent  stakes  and  viewpoints,  and  different  cultures.  In  such  cases,  the  paradigm  of 
“constructivism”  is  more  appropriate  [P  25].  (The  table  below  outlines  the  main  differences  between  the 
two  paradigms.)  Analysing  their  experience,  that  includes  an  experiment  on  the  newly  developed  strategic 
and  operational  planning  process,  the  authors  of  the  report  make  several  observations  that  can  sharpen  the 
understanding  of  both  analysts  and  decision-makers  of  such  specific  cases: 

•  “Truth”  is  a  matter  consensus  among  informed  and  sophisticated  constructed  (it  is  not  just  ‘out 
there’); 

•  “Facts”  have  no  meaning  except  within  some  value  framework,  therefore  there  cannot  be 
“objectivity”; 

•  “Causes”  and  “Effects”  do  not  exist  except  by  imputation  and  all  interacting  parties  are 
responsible; 

•  Phenomena  can  only  be  understood  within  the  context  of  study;  no  generalisation  is  possible; 

•  Treatments  are  not  stable;  they  will  be  affected  as  much  as  the  “dependent”  variables; 

•  Experiment  analysis  produces  data  in  which  facts  and  values  are  inextricably  linked; 

•  Researchers/analysts  are  subjective  partners  with  stakeholders/  audience/  customer  in  the  literal 
creation  of  data. 

Notwithstanding  all  these  features  of  experimentation,  researchers/  analysts  still  have  a  special  role  to 
play. 


Table  1 :  Competing  Paradigms  and  ‘Rigorous’  experimentation. 


Positivism 

Constructivism 

What  is  reality?  (“ontology”) 

• 

Objective,  independent  from  the  observer  • 

Subjective,  dependent  upon  the  observer 

• 

Only  one  • 

Continuously  constructed 

How  to  know  reality?  (“epistemology”) 

• 

Subject-object  duality  • 

Knowledge  is  a  process  of  creation  between 

• 

Exclude  values  from  influencing 

research  and  the  researched 

Methodology 

• 

Control  confounding  variables  and  values  • 

Iterative  process 

• 

Converge  on  truth  • 

Values,  beliefs,  biases  are  included 

• 

Accumulate  validity  • 

Findings  ‘emerge’  from  joint  construction 
of  reality 

5.3 

Roles  of  the  Analysts 

Thomas  Ekstrom  from  the  Swedish  Defence  Research  Agency  (FOI)  identifies  four  types  of  players  in  the 
CD&E  process,  intentionally  exaggerating  certain  features  of  some  typical  attitudes  [P  23]: 
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•  “Artistic  freedom”  of  the  concept  developer,  who  does  not  really  want  a  CD&E  method,  a  CD&E 
process  or  any  other  form  of  CD&E  structure,  relying  instead  on  creativity  and  inspiration; 

•  “Timely  order”  for  the  experimenter,  who  focuses  on  schedule  and  resource  management,  and  not 
on  the  questions  of  interest  to  the  concept  developer; 

•  “Military  relevance”  considerations  for  the  manager  of  the  experimental  platform  -  usually  a 
senior  officer  who  has  to  make  sure  that  the  scarce  resources  at  his  or  her  disposal  are  being  used 
to  develop  the  concepts  in  an  effective  and  efficient  manner; 

•  “Analytical  structure”  considerations  for  the  analyst,  who  looks  that  alternatives  are  explored,  the 
maturity  of  the  concept  is  tested,  that  decision  makers  receive  relevant  information  at  regular 
intervals,  that  concept  developers  are  given  opportunities  to  create  support  among  colleagues 
outside  the  CD&E  community  through  the  dialogue  that  tests  like  seminars,  workshops,  and  war¬ 
gaming. 

Han  de  Nijs  examines  in  detail  the  respective  roles  of  the  analyst  along  the  phases  of  a  CD&E  project, 
with  the  overall  purpose  of  bringing  “structure  and  rigour  in  the  definition  of  problems,  in  methods  and  in 
results.”  This  may  involve  quantitative  or  qualitative  methods.  Most  importantly,  “analysis  brings 
meaning  by  making  better  sense  of  the  world,”  even  when  dealing  with  wicked  problems  [P  21]. 

De  Nijs  further  suggests  that  the  whole  CD&E  community,  as  well  as  other  stake  holders,  would  benefit 
from  a  common  concept  maturity  model ,  analogous  to  the  technology  readiness  levels  used  currently  in 
R&T  management,  a  proposes  a  scale  with  ten  grades  of  concept  maturity  [P  21]. 


6.0  ANALYSIS  OF  CURRENT  OPERATIONS 

The  symposium  session  on  current  operations  addressed  the  effects-based  approach  to  operations, 
intelligence  data  management  requirements,  the  concept  of  tribe  building,  the  factors  contributing  to,  or 
hindering,  coalition  team  performance,  and  lessons  learned — all  at  least  to  an  extent — in  the  context  of  the 
ongoing  operations  in  Afghanistan. 

Analysts  continue  to  address  the  implementation  challenges  of  the  Effects-based  Approach  to  Operations 
and  to  develop  respective  decision  support  tools.  Johan  Schubert  from  FOI  presented  the  Collaboration 
Synchronization  Management  Tool  (CSMT)  used  in  effects-based  planning  and  continuous  evaluation  and 
refinement  of  the  plans  [P  33].  A  Capability  Impact  Matrix  (CIM)  is  used  to  represent  the  influence  of 
activities  of  supporting  effects,  of  supporting  effects  on  “decisive  conditions”,  and  finally  of  decisive 
conditions  on  military  end  state.  The  tool  allows  evaluating  and  representing  alternatives,  as  well  as  the 
consistency  and  the  stability  of  each  instance  of  the  operations  plan.  The  method  takes  subjective 
judgments  about  the  activities  as  input  data  and  to  calculate  assessments  of  all  other  plan  elements.  When 
a  piece  of  evidence  lacks  credibility,  it  is  ‘discounted’  in  the  same  analysis  framework.  Thus,  planners  are 
able  to  identify  not  only  decisive  influences  from  activities,  but  also  to  “find  any  weaknesses  and  all 
strengths  of  the  plan  as  described  by  the  cross  impact  matrix  before  the  effects-based  execution  phase” 
[P  33]. 

To  assess  potential  effects,  however,  planners  need  reliable  intelligence.  Experienced  practitioners  insist 
on  revising  the  intelligence  process  to  support  effectively  Irregular  Warfare  (IW)  and  Stability,  Security, 
Transition,  and  Reconstruction  (SSTR)  operations,  as  well  as  to  reach  decision  makers  [27].  Practitioners’ 
suggestions  have  been  critically  examined  by  analysts  and  presented  to  the  symposium  in  a  report  by 
Michael  J.  Baranick  and  Stuart  H.  Starr  [P  29].  The  authors  explore  the  data  needs  of  analysts  in  support 
of  decision  makers  and  call  for  a  substantial  role  of  the  analytical  community  in  the  intelligence  data 
management  process.  That  focus  is  supported  by  the  finding  of  the  US  Training  and  Doctrine  Command 
Analysis  Center  (TRAC)  that  34  of  35  major  gaps  in  analysis  support  to  IW  can  be  attributed  to  lack  of 
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credible  data.  Several  major  questions  still  wait  resolution,  and  further  studies  may  be  required  to  support 
the  respective  decisions: 

•  How  to  come  to  data  standards  (e.g.,  metadata)  and  a  structure  in  transforming  qualitative  into 
quantitative  data? 

•  How  to  provide  for  a  multidisciplinary,  ‘Whole  of  Society’  discovery  when  needed  data  are 
widely  distributed? 

•  Shall  there  be  a  central  organisation  to  take  responsibility  for  the  data  needed  to  support  analysis 
and  for  developing  needed  ontologies,  metadata,  and  pedigree? 

•  How  to  provide  efficient  verification  and  validation  of  data? 

•  Should  a  Federated  IW  Data  Repository  be  mandated? 

•  What  are  the  suitable  Measures  of  Merit? 

While  complex  adaptive  systems  define  an  emerging  paradigm  in  studying  irregular  warfare  and 
counterinsurgency  operations,  practitioners  need  more  tangible  tools.  Nicolas  Israel  and  Thomas  Peugeot 
search  for  a  formalised  operational  concept  drawing  from  human  sciences,  and  the  anthropology  in 
particular.  Their  studies  suggest  that  tribe  building  can  serve  as  such  concept  to  enable,  inter  alia, 
intelligence  collection  and  data  organisation  and  modelling  and  simulation  efforts  [P  30].  Tribe  building  is 
seen  as  a  set  of  practices  supporting  the  smooth  growth  of  modern  value  spaces  endorsed  by  traditional 
authorities,  and  a  rejection  of  totalitarian  forces.  Such  practices  should  be  performed  with  great  care  due  to 
the  difficulty  of  knowing  a  priori  which  modern  space  should  be  developed  first,  who  the  traditional 
authorities  are  or  what  the  level  of  endorsement  is.  In  line  with  the  findings  of  practitioners  [28],  the 
authors  reason  that  a  successful  strategy  should  be  based  on  stabilization  of  tribes  to  enable  state 
legitimization  and  not  the  opposite.  Studying  the  interaction  between  individual  and  group  behaviour  and 
values,  they  conclude  that  restoring  the  traditional  fabric  of  society  is  the  best  way  to  achieve  peaceful 
transitions  toward  democracy. 

Another  study,  presented  at  the  symposium,  addresses  team  efficiency  in  coalition  settings  [P34]. 
Applying  the  PRISM  (Performance,  Role  interdependence,  Information  Sharing)  model,  the  analysts  study 
the  factors  influencing  information  sharing,  collaboration,  and  ultimately  the  performance  of  coalition 
teams.  They  also  draw  on  the  model  of  Inter-organisational  Collaborative  Capacity  (ICC)  [29].  ICC  is  the 
capability  of  organizations  (or  a  set  of  organizations)  to  enter  into,  develop,  and  sustain  inter- 
organisational  systems  in  pursuit  of  collective  outcomes.  ICC  provides  a  framework  for  understanding 
individual  and  organizational  factors  brought  to  a  newly  formed  team,  and  likely  to  influence  team 
collaboration.  The  authors  apply  the  model  to  study  the  factors  influencing  the  performance  of  U.S.  and 
Bulgarian  teams  training  together  at  the  Novo  Selo  range,  Bulgaria.  The  study  is  expected  to  enhance  the 
means  of  assessing  coalition  team  effectiveness,  or  likelihood  of  success  in  future  NATO  missions.  This 
research  could  be  utilised  to  develop  a  method  of  assessing  the  readiness  of  coalition  team  members  prior 
to  beginning  a  mission  and  to  target  training  to  address  areas  in  need  of  improvement  [P  34]. 

Two  symposium  reports  presented  Lessons  Learned  (LL)  approaches  and  experience  respectively  of 
Bulgaria  [P  03]  and  Norway  [P31].  The  LL  process  benefits  strongly  from  inclusion  of  analysts  in 
operational  units.  Lessons  learned  is  seen  as  a  process  going  beyond  absorption  of  operational  experience 
to  encompass  lessons  from  exercises,  gaming,  and  experimentation,  as  well  as  from  capability 
development  experience.  It  is  recognised  that  people  learn,  not  “organisations”  [P  31].  Therefore,  while 
LL  does  require  the  creation  of  databases  and  implementation  of  advanced  IT,  experts  in  organisational 
culture  and  psychology  are  also  needed  for  effective  and  efficient  operationalisation  of  experience. 
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7.0  MAJOR  OA  THEMES  IN  A  PROCESS  OF  CONSOLIDATION 

There  are  several  transformation-related  analytical  themes  that  cover  two  or  more  areas  in  the  symposium 
programme.  Some  emerged  fairly  recently,  but  are  already  solidly  placed  on  the  agenda  of  both  analysts 
and  users  of  analytical  support.  Hence,  this  section  of  the  report  takes  another  look  at  effects -based 
approach  to  operations,  networks,  and  organisational  development  under  uncertainty. 

7.1  Effects-Based  Approach  to  Operations 

It  is  blatantly  clear,  that  it  is  not  possible  to  achieve  the  end  state  in  a  counter-terrorism  or  counter¬ 
insurgency  operation  by  capturing  a  country’s  capital,  its  territory,  or  killing  a  handful  of  terrorists/ 
insurgents.  We  need  to  be  able  to  tackle  the  influence,  i.e.  the  “hearts  and  minds”  problem.  To  that 
purpose  analysts  should  assist  decision  makers  in  relating  military  resources — and  the  way  in  which  they 
are  used — to  the  outcome  of  peace  support  and  counter-insurgency  operations.  That  requires 
understanding  (modelling)  human  perception  and  behaviour  and  the  ways  by  which  they — individuals, 
crowds,  military  units,  factions,  regimes,  populations,  etc. —  can  be  influenced,  keeping  in  mind  that  there 
is  an  infinite  variety  of  humans  with  their  experience,  beliefs,  priorities,  as  well  as  a  wide  variety  of 
unobservable  contextual  variables.  In  addition,  chance  events  and  outcomes  also  come  into  play  [P  02]. 

While  traditional,  or  ‘hard’,  OA  techniques  still  have  their  place,  in  particular  when  the  problem  at  hand 
can  be  quantified  based  on  physical  data  or  assessment  by  an  adequate  number  of  experts  [P  16],  we  need 
to  enhance  the  capabilities  for  ‘soft’  operational  analysis  [30].  In  the  U.K.  experience,  Dstl  Strategic 
Analysis  Group  is  widening  the  analytical  discipline  base  by  including  psychologists,  anthropologists, 
sociologists  [P  02].  Further,  Roger  Forder  recommends  that  the  analytical  community  needs  to: 

•  Keep  in  touch  with  possibly  applicable  theoretical  approaches,  such  as  complex  adaptive  systems; 

•  Widen  the  use  of  historical/  current  operational  data,  including  broad  conclusions  about  factors 
affecting  success  of  counter-terrorism/COIN  operations  and  factors  affecting  public  support,  as 
well  as  striving  to  achieve  pull-through  from  current  operations; 

•  Use  gaming  to  provide  participants  with  sufficient  ‘wrapping’  to  enable  them  to  play  out  of  their 
own  personality  and  culture  [P  02]. 

The  need  to  enhance  ‘soft’  analytical  power  is  confirmed  by  other  studies  as  well.  For  example,  in 
studying  intelligence  data  gaps,  it  was  concluded  that  20  of  the  35  identified  gaps  required  at  least  some 
“soft  science”  solutions  [P  29]. 

Such  findings  are  of  benefit  to  the  operations  planning  process,  to  capabilities-based  planning,  CD&E,  and 
training  among  the  areas  of  highest  importance  to  defence  transformation. 

7.2  Effective  Networking 

The  effect-based  approach  has  to  be  applied  comprehensively  in  dealing  with  the  opponents,  their  support 
base,  and  contextual  factors,  as  well  as  own  support  base.  The  principle  of  comprehensiveness  applies  also 
to  the  use  of  all  available  means  to  resolve  a  particular  crisis  situation,  whether  they  are  ‘owned’  by 
military  forces,  intergovernmental  organisations,  NGOs,  private  security  companies  or  other  businesses, 
that  act  in  a  theatre  as  a  network. 

Comprehensive  approach  is  not  just  a  concept,  but  a  practical  challenge  that  requires  design  and 
application  of  adequate  concepts,  methodologies  and  tools.  Concepts  like  3D  (Defence,  Diplomacy, 
Development  aid),  Whole  of  Government  approach  and  other,  developed  by  policy  experts,  are  finding  a 
way  into  the  security  and  defence  planning  communities.  The  experience  of  the  U.S.  Department  of 
Homeland  Security  shows  that  best  practice  in  capability -based  planning  can  be  amended  and  modified  to 
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the  needs  of  other  security  sector  organisations,  and  the  approach  has  been  found  of  utility  by  other  non¬ 
defence  actors  such  as  the  World  for  Food  Programme  [19]. 

The  challenge  of  transferring  effects-based,  capability  planning  and  capability  delivery  models  and 
methods  to  other  actors  in  the  network  is  not  one  of  hard  analytics.  Since  there  is  no  single  planning 
‘centre,’  ‘soft’  approaches  would  contribute  to  finding  suitable  arrangements  in  planning  the  response — 
both  in  preparing  and  maintaining  ‘capabilities’  and  in  the  coordination  of  in-theatre  activities.  ‘Social 
media’ -type  of  platforms,  accompanied  by  adequate  soft  analysis  methodologies  and  tools  may  prove  the 
usefulness  of  the  analytical  community  in  an  inter-agency  and  broader  international  setting. 

7.3  Organisational  Development  Under  Uncertainty 

The  third,  and  final  theme  addressed  in  this  report  is  how  to  manage  the  evolution  of  defence 
organisations,  including  the  military,  in  the  face  of  uncertainty.  One  emerging  approach  calls  for  making 
defence  organisations  agile  by  design,  i.e.  to  make  them  responsive  and  able  to  adapt  quickly  to  changing 
threats,  new  types  of  operations,  technological  opportunities,  funding  levels,  shifts  in  socio-demographic 
environment,  etc.  On  the  other  hand,  it  takes  years  to  fully  develop  most  types  of  defence  capabilities,  and 
the  decision  making  process  in  NATO  and  its  member  nations,  being  under  the  scrutiny  of  parliaments  and 
society,  is  rather  slow  compared  to  the  dynamics  of  change  we  currently  witness.  This  dilemma  of 
Stability  versus  Agility  of  defence  organisations  has  yet  to  be  resolved,  but  some  promising  ideas  are 
already  put  forward  to  the  symposium  discussion. 


8.0  CONCLUSION  AND  RECOMMENDATIONS 

The  symposium  brought  together  90  analysts,  decision  makers  and  military  officers  from  19  NATO  and 
partner  countries  to  discuss  the  major  transformation  themes  of  capability  planning,  capability 
development,  concept  development  and  experimentation,  counter-terrorism/  counter  insurgency  operations 
and  lessons  learned.  One  important  topic,  however,  was  missing  from  the  symposium  agenda  -  the 
analytical  support  to  advanced  training. 

A  number  of  other  topics  were  also  addressed  repeatedly  by  presenters,  as  well  as  during  the  discussion. 
The  following  were  among  them: 

•  What  are  the  appropriate  roles  of  the  analyst  in  the  current  wave  of  defence  transformation? 

•  What  is  necessary  to  make  the  exchange  between  analysts  and  decision  makers  effective? 

•  What  are  the  best  ways  to  organise  the  analytical  support? 

•  How  to  make  alliance  and  national  defence  planning  processes  transparent,  traceable,  and 
auditable? 

•  How  to  arrive  at  a  common  lexicon,  e.g.  on  capability,  lines  of  development,  capability  taxonomy, 
task  lists,  generic  units,  etc.? 

•  How  to  define  good  measures  of  effectiveness  and  efficiency? 

•  How  to  make  cost-  and  performance-related  benchmarking  studies  more  effective  and  efficient? 

•  What  are  the  appropriate  methodologies  for  managing  knowledge  related  to  defence 
transformation? 

•  How  the  analytical  community  can  contribute  to  enhancing  integrity  and  reducing  corruption  risks 
in  defence? 

•  How  to  make  defence  organisations  sufficiently  agile  in  the  face  of  contradicting  requirements? 

•  Are  there  emerging  transformation  themes  that  require  new  types  of  analytical  expertise? 
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In  addition  to  the  suggestions  made  throughout  the  text  of  this  report,  we  can  make  several 
recommendations,  the  implementation  of  which  would  enhance  the  analytical  support  to  defence 
transformation.  A  number  of  studies  may  be  beneficial  in  that  respect: 

•  Organisational  development  under  uncertainty .  How  to  deal  with  the  contradicting  requirements 
of  decision-making  processes  in  democracy  and  long  lead  times  for  defence  systems,  on  one  hand, 
and  the  need  to  react  quickly  to  unforeseen  changes  in  the  force  development  environment? 

•  Analytical  support  to  training.  How  to  make  joint  and  multi-agency  training  more  effective  and 
efficient? 

•  Taxonomy  of  OA  ‘ technologies’  (approaches,  methods,  and  tools).  What  is  the  spectrum  of 
analytical  methods  and  tools  applicable  to  defence  issues?  A  taxonomy  that  is  commonly  used  by 
the  NATO  analytical  community  would  facilitate  benchmarking  studies,  potential  role 
specialisation  among  nations  (just  like  the  capability  specialisation  of  the  armed  forces  of  NATO 
nations)  and  cooperation  among  nations,  thus  contributing  to  the  efficiency  of  analytical  support 
to  defence  transformation. 

To  contribute  effectively  to  defence  transformation  and  receive  recognition,  the  analytical  community  of 
NATO,  NATO  nations  and  committed  partners  needs  to  focus  on  providing  organisational  agility,  proving 
the  efficiency  of  defence  organisations,  and  of  analytical  support  itself,  and  strengthening  partnerships. 
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